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Abstract 

By 2025, U.S. air traffic is predicted to 
increase three fold and may strain the current air 
traffic management system, which may not be able 
to accommodate this growth. In response to this 
challenge, a revolutionary new concept has been 
proposed for U.S. aviation operations, termed the 
Next Generation Air Transportation System or 
“NextGen”. Many key capabilities are being 
identified to enable NextGen, including the use of 
data-link communications. Because NextGen 
represents a radically different approach to air 
traffic management and requires a dramatic shift in 
the tasks, roles, and responsibilities for the flight 
deck, there are numerous research issues and 
challenges that must be overcome to ensure a safe, 
sustainable air transportation system. Flight deck 
display and crew-vehicle interaction concepts are 
being developed that proactively investigate and 
overcome potential technology and safety barriers 
that might otherwise constrain the full realization of 
NextGen. The paper describes simulation research, 
conducted at National Aeronautics and Space 
Administration (NASA) Langley Research Center, 
examining data-link communications and traffic 
intent data during envisioned four-dimensional 
trajectory (4DT)-based and equivalent visual (EV) 
surface operations. Overall, the results suggest that 
controller pilot data-link communications (CPDLC) 
with the use of mandatory pilot read-back of all 
clearances significantly enhanced situation 
awareness for 4DT and EV surface operations. The 
depiction of graphical traffic state and intent 
information on the surface map display further 
enhanced off-nominal detection and pilot 
qualitative reports of safety and awareness. 

Introduction 

A key enabler of envisioned operations under 
NextGen and its “net-centric” environment is data- 
link communications and a voice-by-exception 
airspace environment. The research literature 


suggests that data-link may improve one source of 
miscommunications - the inability to get the 
message from one party to the other - but it does 
not necessarily address the rest of the 
communications process - i.e., whether the message 
was understood and whether it accurately conveyed 
the speaker’s intent. In a NextGen environment - 
with closer spacing and more pilot responsibility for 
separation - increased head-down time and 
increased workload could significantly reduce 
safety margins. Concerns have also been raised in a 
net-centric data-link environment on the need or 
modality change effects associated with the 
presence or absence of “party-line” 
communications [1] — [15]. While past work notes 
its criticality, these works also show that the “party- 
line” information influence is dependent upon crew 
resources, communication, and workload. These 
influences could possibly be impacted by 
generational versions of NextGen - for example, 
the potential that ADS-B “in” capability, possibly 
augmented with intent data, might provide a 
suitable alternative to party-line voice 
communications. Because NextGen represents a 
radically different approach to air traffic 
management, NASA is addressing numerous 
research issues and challenges that must be 
overcome to ensure a safe, sustainable air 
transportation system. 

In this study, research specifically addressed: 
1) safety concerns regarding changes in 
communication modalities and 2) the impact of 
emerging NextGen operational concepts and 
possible crew-vehicle interface mitigations to 
enable operations, such as equivalent visual 
operations (EVO), or four-dimensional trajectory 
(4DT)-based air and surface operations [16]- [17]. 

Data-Link Communication 

The challenge for NextGen is to replicate the 
strengths and advantages of the previous 
communication method (i.e., voice) while bringing 



to bear the benefits of the new. NASA is 
conducting a research portfolio which strives to 
meet this goal. Part of this work targets flight crew 
assessment of the modality change effects 
associated with data-link operations, the impact of 
party-line information changes, and potential 
mitigating or confounding factors for NextGen 
operations. Past research has demonstrated that the 
efficacy of party-line information is dependent upon 
crew resources, communication, and workload. 
These influences are undoubtedly also impacted by 
auditory demands and other visual data available to 
the flight crew. In particular, what potential exists 
in the information available by use of ADS-B “in” 
capability, with traffic intent data? Could the visual 
display of data-link information and associated 
navigation and intent data serve to replace voice 
(voice-by-exception) for routine communications 
and compensate for the potential impacts of party- 
line information loss or inherent latencies in 
managing mixed voice and data pilot- Air Traffic 
Control (ATC) communications with Controller- 
Pilot Data Link Communications (CPDLC). 

To achieve the NextGen vision, research is 
necessary to proactively investigate technology 
which may overcome potential safety barriers that 
would otherwise constrain surface traffic 
optimization, maximum runway capacity through 
reduced runway occupancy time, simultaneous 
single runway operations, and 4DT and equivalent 
visual operations, among others [16]-[17]. These 
safety objectives were the focus of a research study 
conducted at NASA Langley Research Center that 
attempted to examine a set of safety issues 
associated with data-link communications with 
surface operations being envisioned for NextGen. 

The data-link communication experiment 
evaluated how data-link implementations and 
mixed-modality communication (i.e., voice and 
data-link) impact crew decision-making, workload, 
situation awareness, and crew coordination during 
operational situations conducted under envisioned 
4DT EVO NextGen operational concepts. This 
paper specifically describes the off-nominal 
performance aspects of this research and 4DT taxi 
performance. Off-nominal scenarios (e.g., incorrect 
clearances, pilot blunders/errors, and data-link 
transmission errors) were unexpectedly introduced 


as critical tests of the robustness and safety of the 
operations. 

Traffic Intent Display 

A second experiment examined various levels 
of traffic intent information on a surface map 
display. In this experiment, intent is operationally 
defined as aircraft state and assigned 4DT route 
guidance information. The experiment assessed 
whether intent information may aid or be required 
in NextGen operations and to what extent or what 
instantiations might be required. The paper 
describes the off-nominal detection results only. 
Off-nominal event testing was included, testing 
surface conflicts and taxi blunders, incorrect data- 
link clearances, detection of non-participating 
objects, and runway incursions. 

METHOD 

Research Flight Crews 

Twenty-four pilots served as participants for 
the research, representing twelve flight crews. Each 
flight crew flew for a major U.S. air carrier and was 
paired by airline to ensure crew coordination and 
cohesion with regard to terminal and surface 
operational procedures. The flight crews were 
provided with a forty-five minute briefing to 
explain the display concepts and evaluation tasks. 
After the briefing, a 1-hour training session was 
conducted to familiarize the flight crews with the 
simulator, the simulated airport, published arrivals 
and departures, display concepts, and evaluation 
tasks. The flight crews were asked to maintain 
good crew resource management and comply with 
company-specific standard operating procedures. 

Simulation Facility 

This research was conducted in the Research 
Flight Deck (RED) simulator at NASA Langley 
Research Center (Figure 1). The simulated aircraft 
dynamics model was a medium- to long-haul 
commercial passenger aircraft. The RFD 
configuration is a fixed-based, dual-pilot simulator 
with a collimated 200° panoramic out-the-window 
scene. Operations were conducted at the Chicago 
O’Hare airport (KORD). The out-the-window 
scene included realistic taxiways and runways with 
appropriate markings, airport lighting, and other 



aircraft in various simulated weather/lighting 
conditions. The visual acuity of the out-the- 
window scene was 20/40 (pixels per degree). The 
RFD was equipped with a 30 0 Horizontal x 24 0 
Vertical Head-Up Display (HUD) located on the 
left or captain’s side. 

As shown in Figure 1, the simulator has four 
large main instrument panel displays referred to as: 
(left to right) Pilot’s Primary Flight Display (PFD), 
Pilot’s Navigation Display (ND), Co-pilot’s ND, 
and Co-pilot’s PFD. The four display panels are 
17” Liquid Crystal Displays with 13.25” x 10.5” 
viewable space at 1280x1024 (SXGA) resolution 
and a 5:4 aspect ratio. The RFD includes a Mode 
Control Panel (MCP), Flight Management System 
(FMS), Control Display Units (CDU), and 
hydraulic side-stick control inceptors (Figure 1). 



Figure 1 - Research Flight Deck Simulator 


Two Electronic Flight Bags (EFBs) were 
installed, one for each pilot. The EFBs are mounted 
above and outboard of the side-stick control 
inceptors near the side window sill. The EFBs 
utilize a dynamic menu interface using the line 
select buttons or touch-screen operation. Custom 
software was developed to provide desired 
experimental functionality, including CPDLC, 
airport maps and charts. The airport map page and 
charts pages displayed standard FAA diagrams that 
enabled the pilots to zoom and pan utilizing 
available buttons on the interface. 

Display Formats 

Flight deck display concepts were developed 
to evaluate how data-link may be implemented for 
NextGen and how best to optimally display that 
information to the flight-crew. In the following, 


candidate flight deck display concepts are shown. 
These concepts have been designed, based on 
current state-of-the-art flight deck display concepts 
(Boeing 787, Airbus A350, & Gulfstream G550) 
and modified for the study to support terminal area 
and surface operations. These candidate solutions 
were evaluated against the envisioned NextGen 
issues and instantiations of data-link. 



Figure 2 - Primary Flight Display 


Primary Flight Display 

The PFD, ND, and Engine Indication and 
Crew Alerting System display (EICAS) were 
modeled after Boeing 787, Gulfstream G550 and 
Airbus A350 instrument panels as current state-of- 
the-art production aircraft. The PFD display unit 
includes an ATC message area (CPDLC message 
area; Figure 2) on the outboard third of the display 
unit showing incoming and outgoing ATC data-link 
communications in alpha-numerical format. 
Incoming messages are color-coded green while 
outgoing messages are white. All messages are 
time-stamped. The captain’s inboard display unit 
shows navigation and EICAS displays. 


Navigation Display 

The first officer’s inboard display is a full- 
screen moving map ND with high airport surface 
detail. Unlike today’s equipage, surface and 
airborne traffic icons were included to simulate an 
ADS-B-In (Automatic Dependent Surveillance 
Broadcast) environment. This display 

automatically transitions to the surface map display 
(described below) after landing, when the ground 
speed is less than 80 kts (Figure 3). 






Figure 3 - Navigation Display 



Figure 4 - Electronic Flight Bag 

Electronic Flight Bag Display 

The EFB, shown in Figure 4, is used as the 
flight crew’s interface for ATC data-link 
communications, approach chart and airport 
diagram chart references. The CPDLC/ATC 
functionality was modeled from existing 
commercial aircraft vendor interfaces. 

Head-Up Display 

The HUD surface operations concepts, Figure 
5, evolved from Taxiway Navigation and Situation 
Awareness (“T-NASA”) research [18] and Runway 
Incursion Prevention System research [19]. The 
HUD showed current ground speed in digital 
format, the current taxiway, next cleared taxiway, 


and centerline markers for the cleared route 
(CPDLC-assigned route). 



Figure 5 - Heads-Up-Display Concept 


In addition, the HUD format included 
modifications to accommodate emerging NextGen 
4DT surface operations. Speed- or time-based taxi 
guidance was calculated and explicitly shown on 
the HUD. The 4DT speed and time symbology 
guided the flight crews throughout the taxi route to 
regulate and meet the required precision of 4DT 
surface operation. 

Surface Map Display 

The nominal surface map display, Figure 6, is 
an enhanced version of the track-up navigation 
display. Own-ship position is indicated by the large 
white chevron. The surface map is automatically 
displayed on the ND during the landing rollout or it 
can also be manually selected at any time. 

The surface map display enhancements were 
experimentally added as part of the studies. These 
enhancements may help to enable 4DT compliance 
and situation awareness. The surface map display 
designs largely reflect current RTCA SC- 183 
working group findings in regard to element shape 
and color assignments, and is designed for strategic 
use for surface operations (to minimize head-down 
time). For instance, when surface traffic is included, 
medium tan colored chevrons are shown on the 
display when on the surface and cyan when 
airborne. When the cleared taxi route is graphically 
shown, it is drawn in magenta for own-ship. 





Figure 6 - Surface Map Display 

Additionally, the alphanumerical display of 
4DT guidance and traffic status/intent data was 
added to aid in 4DT surface operations. For 
instance, one enhancement was a list of aircraft 
position and associated state data (obtained from a 
modeled ADS-B, TIS-B, and CPDLC-all 
surveillance source). This traffic list is shown on 
the right side of the display, sorted by proximity to 
own-ship with the closest traffic listed first. A 
cursor control device can be used to select specific 
traffic in the list (magenta box indicates selected 
traffic) which brings up additional details of the 
selected traffic (lower right of Figure 6). Traffic 
details included: type, flight ID, speed, heading, 
ATC assigned route (intent), and the range / bearing 
from own-ship position. Own-ship-assigned taxi 
route could be displayed on the lower center of the 
display as a text string, and current position along 
that route is highlighted in magenta text. Distance 
to the next route segment is also displayed nearby 
in green text. 4D trajectory guidance information 
(textual) could also be displayed textually on the 
bottom of the ND: current state information in 
white [ground speed, elapsed time, and Estimated 
Time-of- Arrival (ETA) UTC]; while guidance 
information [Required Time of Arrival (RTA) in 
Greenwich Mean time (UTC), Required Time En- 
route (RTE) in minisec] was shown in magenta. 
Actual ground speed in white and guidance ground 
speed to meet RTAs was displayed textually in 
magenta in the upper left hand comer of the ND. 

For the advanced traffic intent surface map 
display concepts, when selected by the flight crew, 
a traffic’s chevron was highlighted with a white 


halo and the intended (ATC assigned) route was 
graphically depicted on the map in the traffic icon 
color (see Figure 7). This graphical routing 
information was augmented with a visual depiction 
of 4DT guidance. A guidance cue - a magenta dot - 
was shown on the commanded route, moving along 
the ATC-assigned route approximately 30 seconds 
ahead - and an own-ship speed symbol - a white 
circle - showing own-ship position estimated in 30 
seconds from present speed and acceleration. The 
flight-crew’s task was to overlay the 30 second 
speed trend symbol (circle) over the 30 sec RTA 
prediction symbology (dot). The same indications 
of traffic intent were shown for selected traffic. 
The symbology is designed such that conflict areas 
(such as crossing routes) can be quickly seen and 
interpreted at a glance. Further, the 30 second 
ahead gives strategic planning information which 
could aid in possible traffic conflicts. End-of-route 
symbology was also shown using a white and 
magenta diamond to indicate relative ETA and 
RTA graphically. 



Figure 7 - Advanced Surface Map Display 


Experiment One Conditions 

Experiment One was designed to evaluate how 
data-link implementations and mixed-modality 
communication (i.e., voice and data-link) impact 
crew decision-making, workload, situation 
awareness, and crew coordination for possible 
NextGen (circa 2008 through 2025) operational 
situations. To reflect evolutionary applications of 
data-link and CPDLC, the research examined four 
communication modes that have been identified to 
represent the gamut of potential technological 
implementations: 







• Data-Link + Voice : The voice + data-link 
condition (where data-link is redundant to voice) 
required the pilot-not-flying to respond via both 
voice and data-link to ATC clearances. All ATC 
clearances were provided by both voice and 
CPDLC and heard on the VHF (“party-line”) 
broadcast. This condition retains the benefits of 
voice and adds the capability for retention and 
review of the message in the cockpit. However, 
this method may significantly increase the workload 
of ATC and the flight crew and does not reduce the 
VHF bandwidth. 

• Data-Link + Read-back Only : The voice + 
data-link condition (where data-link is supplemental 
to voice) stipulated that pilots provide radio read- 
backs on the VHF channel of received data-link 
messages. ATC sent all clearances via data-link 
messaging and all flight crews were required to 
respond by voice (a condition in which CPDLC was 
supplemental to voice). This condition reduced the 
VHF chatter by 50% (except for ATC non-routine 
communications) and retains the retention/review of 
the message in the cockpit. This method also 
retains the “read-back” as a checking mechanism as 
to whether the communication was correctly 
received and understood. The advantage of this 
method is that the communication frequency is 
reduced but possibly not sufficiently, and ATC 
must use both voice (listening) and data-link 
concurrently. 

• Data-Link Only : The “data-link only” 
condition (i.e., data-link as replacement for voice or 
voice-by-exception) had no ATC clearances or pilot 
read-backs over the voice channel and the 
participants were required to respond only through 
data-link (except for exigent situations or non- 
routine communications). This technique provides 
the maximum VHF bandwidth reduction and 
simplifies the communication to a single modality. 
However, this condition also represents the loss of 
the auditory communication benefits (party-line, 
intonation, stress, etc.) 

• Data-Link + Intent : The data-link only + 
traffic intent condition was identical to the data-link 
only condition (i.e., data-link as replacement for 
voice) but presented additional traffic intent 
information (e.g., selected traffic route and 
destination information). The condition was 
introduced to determine whether providing traffic 


intent to the flight crew, shown on the surface map 
display (see Figure 8), may mitigate safety issues 
described by other research studies as a 
consequence of reduced voice communications and 
loss of party line information. 


Experiment Two Conditions 


For this experiment, intent is operationally defined 
as the information that collectively provides own- 
ship and other aircraft state and assigned 4DT route 
guidance and prediction (e.g., ADS-B in/out surface 
message; CPDLC-all; TIS-B; etc). The objectives 
of Experiment Two were to determine what critical 
elements are needed to enable EVO and 4DT 
surface operations efficiently but above all, safely. 
Essentially, the questions being asked are: whether 
it is sufficient to just present the traffic intent 
information textually (e.g., data-link clearances of 
other aircraft); or, 2) is it required to have a 
graphical interface of traffic intent display. 



Figure 8 - Own-ship Route & Traffic 


For this experiment, the display conditions were: 

• A baseline NextGen aircraft capability to include 
a surface map display with own-ship only and 
CPDLC capability with textual traffic state 
information (see Figure 6); 

• An intermediate NextGen concept to include a 
surface map display with traffic and manual query 
capability of other aircraft intent and graphical 
depiction of own-ship, target aircraft, and 4DT 
paths (see Figure 8); 




• An advanced NextGen concept that automatically 
prioritizes and selects aircraft while depicting 
graphical own-ship and traffic 4DT guidance and 
conformance (see Figure 7). 

Experimental Design 

The results presented herein were part of a 
larger research study consisting of three 
experiments that were conducted over two-days of 
testing for each flight crew. The experiments ran 
sequentially and the first two experiments are 
described here. The experimental design was a 
mixed-subjects factorial design. The independent 
variables of interest were communication mode 
(Experiment One) or surface map display concept 
(Experiment Two) for nominal (within-subjects) 
and off-nominal trials (between-subjects). All 
flight crews experienced each of the four off- 
nominal trials but the communication mode (4 
levels) or display concept (3 levels) for each trial 
was randomly varied across the participants. 
Therefore, each flight crew saw all four off-nominal 
events but with each of the different communication 
modes or display concept such that across all flight 
crews there was a balanced presentation of off- 
nominal events paired with communication mode or 
display concept dependent upon experiment. 

All 20 nominal trials were presented in the 
same order to the flight crews but was randomly 
ordered in which communication mode was paired 
with that trial. The nominal trials provided the 
flight crews with experience with the various 
communication modes and the varying geometries 
of terminal maneuvering area procedures available 
at KORD. The nominal trials also served to 
increase the total number of trials to provide the 
necessary experimental matrix to introduce the off- 
nominal trials and to best replicate the operational 
conditions most likely to produce errors. 

The dependent variables included quantitative 
and qualitative measures. The quantitative 
measures included 4DT adherence and 
conformance (e.g., commanded and actual speed 
delta), flight technical error (e.g., root-mean- 
squared-error of aircraft position from taxi 
centerline), pilot control inputs (e.g., toe brake 
force, control wheel column and tiller inputs), off- 
nominal detection and response, and flight crew 
communication acts [21]. The qualitative measures 


included Likert-type post-run questionnaires, 
Situation Awareness Rating Technique, and NASA 
Task Load Index, and post-experiment paired 
comparisons for Situation Awareness (SA) - 
Subjective Workload Dominance (SWORD). 

Rare Event Scenarios 

An overview of the off-nominal rare event 
scenarios are presented in the following. 

Erroneous Data-link Clearance. The first 
off-nominal scenario was created by an incorrect 
data-linked clearance that produced a route 
disconnect. The erroneous data-link inclusion 
would result in the aircraft turning back toward the 
active runway instead of toward the concourse 
creating a potential runway incursion situation. 

The data link clearance was in dissociation 
with the voice clearance (for those conditions that 
had voice) in that the clearance was stated correctly 
by the controller over the voice channel. Therefore, 
the flight crew could detect the anomaly if they 
heard and understood the dissociation between the 
voice and data-link clearances or by reviewing the 
data-link clearance fully prior to acknowledging 
comply with WILCO (or later during taxiing). It 
was hypothesized that voice (as redundant to data- 
link) would result in greater awareness and 
likelihood of detection of the error because the 
flight crew would have more opportunities to 
realize the error. 

Data-Link Clearance Received for Other 
Aircraft. The second off-nominal scenario 
examined the effect of erroneously receiving a data- 
link clearance intended for another aircraft. The 
scenario began with an approach to Runway 22R. 
The proper data-link clearance was received prior to 
the final approach fix containing the correct landing 
and taxi routing data. Approximately 0.5 nm from 
the runway threshold, the aircraft received a second 
landing clearance that was erroneously sent to the 
aircraft although it was intended for another aircraft 
conducting operations on a proximate runway. The 
clearance contained a similar taxi clearance as the 
previous CPDLC message but also had errors that 
would result in a route dissociation between the 
landing runway and the taxi instructions. 

The erroneous message was hypothesized to be 
generated by a controller error where the message 



was sent to and contained a transposed aircraft call 
sign (which was similar to the own-ship aircraft). 
For those conditions that included voice, the voice 
clearance was accurate and correctly stated the 
intended aircraft call sign and taxi clearance. The 
hypothesis was that, during the high workload 
phase of the approach when the incorrect clearance 
was received, the flight crew would not be able to 
attend to the CPDLC message and would accept the 
clearance without review. This contrasts with the 
flight crews who also heard the voice clearance, 
thus, providing them an additional opportunity to 
recognize that the taxi clearance was not congruent 
with the landing runway and was not intended for 
their aircraft. 

Taxiway Conflict. The third off-nominal 
scenario evaluated whether the flight crew would be 
task-captured by the 4DT task. The scenario began 
as a taxi-out task in which the aircraft received push 
back and taxi clearance to the departure runway 
with RTA instructions. Once the aircraft accepted 
the clearance, another aircraft subsequently landed 
and turned off with a taxi clearance to Delta. The 
conflicting aircraft communicated on tower 
frequency, that they were unsure where Taxiway 
Delta was. The conflict aircraft mistakenly turned 
on the Taxiway Mike instead of the second taxiway 
(Delta) resulting in a potential nose-to-nose 
situation. 

The objective of the scenario was to determine 
if any of the experimental conditions would result 
in significant head-down time and increased intra- 
cockpit communications that may prevent the 
timely identification of the other traffic. (The 
scenario was not intended to result in a collision but 
would have resulted in a nose-to-nose situation 
requiring ground vehicles to tow the aircraft if not 
detected in an adequate amount of time.) The 
hypothesis was that the flight crew that had either 
of the voice and data-link conditions would more 
easily recognize the potential conflict and would be 
proactive in avoiding the situation because of the 
reduction in hypothesized intra-cockpit 
communication and reduced head-down time. The 
flight crews that had only data-link (e.g., data-link 
only and data-link + intent) would instead be 
reactive in detecting the situation because of 
increased communication and head-down time 


involved in interacting with the EFB and surface 
map displays. 

Approach Runway Incursion. The final off- 
nominal scenario for Experiment One (data 
communication) created a potential runway 
incursion in which an aircraft erroneously taxied 
onto the runway and held despite it being cleared 
for take-off while the subjects were on final 
approach to the same runway. The blundering 
aircraft had responded to ATC, but did not correctly 
acknowledge the clearance, and the controller failed 
to recognize the error (expectancy heuristic). The 
aircraft consequently remained on the runway, 
unaware that they were supposed to depart because 
of the aircraft on approach behind them. 

It was hypothesized that having party-line 
information would cue the flight crew that an 
aircraft was holding on the landing runway. Also, 
having data-link with traffic intent might also alert 
the flight crew to the runway hazard. 

ADS-B Surveillance Failure. An off-nominal 
scenario for Experiment Two, examining the 
display and use of traffic intent data received by 
aircraft, evaluated pilot behavioral response to a 
surveillance source failure. Own-ship was taxiing 
in low-visibility conditions (1200 RVR). However, 
traffic was displayed on the surface map (either 
textually and/or graphical icons) and in the FLIR on 
the HUD. After turn-off from the active runway, 
the aircraft taxiing in front of own-ship lost the 
capability to broadcast their intent or positional 
information. At this time, the aircraft was removed 
from the traffic list and, if available, the traffic icon 
representing that aircraft disappeared from the 
surface map display. The scenario was designed to 
test whether the flight-crew would notice the 
absence of critical information in the surface map 
display. 

Taxi Aircraft Blunder. The next off-nominal 
scenario for Experiment Two involved a runway 
incursion scenario. Another aircraft taxied into 
position for an intersection departure when ATC 
had already cleared own-ship for departure. This 
scenario evaluated if and how the flight crew were 
alerted to this unsafe condition, either by: (a) ATC- 
pilot voice communication (present in all 
Experiment Two scenarios); (b) detecting the 
aircraft on the surface map; or (c) seeing the aircraft 
in the HUD FLIR during take-off roll. 



Non-Participating Vehicle on Surface. A 

significant concern for current and future surface 
operations is the prevalence of vehicular traffic that 
is non-participating in communication, navigation, 
or surveillance in the airport movement area. This 
scenario was designed to test whether the flight 
crews would get complacent and rely on the surface 
map display of traffic information and not 
adequately scan for other traffic and vehicles. The 
flight crews were aware of the presence of 
significant vehicle traffic, representing typical 
KORD operations, and that these vehicles were not 
under positive surveillance by ATC or position 
broadcast to own-ship. During this scenario, a fire- 
truck was taxing on the ramp area but got lost and 
notified ATC that they may be on a taxiway. The 
fire truck was in the grass island between two 
taxiways; however, the wingspan of the 757 would 
clip the top of the fire truck. Because the fire truck 
was broadcasting its position, it was not iconically 
shown on surface map but was visible with FLIR 
and OTW. The scenario was designed to test 
whether the HUD near-domain information (e.g., 
symbology) would task capture the pilot taxiing 
and/or whether the surface map displays would 
induce reductions in OTW scanning. 

Controller Mistake Clearing Own-ship 
Across Active Runway. The final off-nominal 
scenario tested in Experiment Two involved a 
similar scenario as the taxi aircraft blunder except 
that own-ship was set-up to cause the incursion. In 
this scenario, own-ship was cleared to cross the 
active runway at which time another aircraft was 
cleared into position-and-hold for take-off. As 
own-ship was approaching the hold short line, the 
tower cleared the other aircraft for take-off. This 
scenario evaluated if and how the flight crew were 
alerted to this unsafe condition, either by: (a) ATC- 
pilot voice communication (present in all 
Experiment Two scenarios) or (b) detecting the 
aircraft on the surface map. 

Experimental Procedure 

The flight crews were recruited from a NASA- 
established protocol. Each crew was given detailed 
pilot briefing materials prior to their arrival. All 
flight crews completed an informed consent and 
were provided a detailed briefing by the researchers 
upon their arrival. Participants were made aware of 
the experimental variables under test but were not 


cognizant that off-nominal events would be 
presented (this was confirmed in post-experimental 
debriefings). Three briefings in-total were 
presented over the two days of testing with each 
briefing focusing on the specific research 
objectives. A 90-minute training session was 
conducted prior to the start of each experimental 
phase, and pilots were trained to an asymptotic or 
equivalent level of performance based on researcher 
observations and pilot feedback. 

Experiment One: Data Communication 

After training, twenty-four trials (20 nominal trials 
and 4 off-nominal trials) were conducted with the 
appropriate breaks included for each day of testing 
(i.e., data communication and traffic intent 
experiments). Prior to each trial, the pilots were 
briefed on the evaluation task (e.g., approach to 
Runway 10) and were provided time to configure 
the aircraft for the operation (i.e., approach, 
departure, or taxi) and brief the procedure. Once 
the flight crew indicated they were ready, the trial 
began including a data-linked CPDLC message that 
was announced with a chime and displayed on the 
PFD CPDLC window and the EFB communication 
sub-menu. The First Officer (FO) reviewed and 
acknowledged the message through the EFB 
(through a quick key access on main menu). 
Dependent upon the evaluation task, the trials lasted 
between 5 and 10 minutes. Pilots were asked to 
adhere to the speed and time commanded guidance 
to meet the RTA within a defined tolerance (+/- 5 
kts and +/-30 sec), but to note any anomalies and to 
ensure safety of the aircraft at all times. After the 
trial ended, each pilot was asked to fill out several 
questionnaires that were administered on selected 
runs (12 experimental runs) that represented a 
cross-section of the nominal runs or were nominal 
analogs of the off-nominal trials. 

Experiment Two: Traffic Intent. The 

experimental procedures for Experiment Two, 
which evaluated various surface map and traffic 
intent display concepts, were identical to 
Experiment One. The notable exception was the 
four different off-nominal events that were inserted 
in the experimental matrix. These off-nominal 
events were fundamentally different than the off- 
nominal events in Experiment One and pilot de- 
briefing confirmed that they were undetected. Full 
data-link and voice communications (data-link + 



voice) were used in the experiment to avoid 
confounds and to provide a consistent set of 
environmental conditions across flight-crews and 
display conditions. 

Experiment One Results 

The experiment, as a whole, had numerous 
objectives. The focus here is on the off-nominal 
results and qualitative data that reflect the efficacy 
of communications and display conditions to detect 
and recognize errors with flight-deck data-link 
CPDLC communications and off-nominal event 
detection with the various surface map display 
concepts. Quantitative data was collected for the 
nominal trials and is presented elsewhere [20]. 

All significant results are presented using an a 
priori significance (a) level of 0.05. Analyses of 
Variances (ANOVAs) were performed when 
appropriate Multivariate Analysis of Variance 
(MANOVA) tests were found to be significant. No 
significant results were found between Captain and 
FO qualitative ratings (p > 0.05) and, therefore, are 
grouped for analyses. 

4DT Conformance and Taxi Performance 

An ANOVA found no significant main effects 
across the four communication modes for cross- 
track root-mean-squared error, F(3,94) = 0.867; 
ground speed deviation from commanded 4DT 
ground speed, F (3,94) = 0.748; and time deviation 
from RTA for the nominal analog or equivalently 
designed trials, F(3,94) = 1.180. Generally, the 
flight crews ranged from 16.45 to 27.45 Root Mean 
Square (RMS) error and maintained speeds within 
1 .5 knots of commanded groundspeed allowing the 
aircraft to conform to RTA times within 2.5 
seconds of assigned RTA. Therefore, these results 
demonstrate that flight-crews are able to maintain 
ground speed commands and adhere to required- 
time-of-arrival clearances. 

Situation Awareness 

Situation Awareness Rating Technique 
(SART). The SART uses the constructs of: 1) 
demand on attentional resources; 2) supply of 
attentional resources; and, 3) understanding. Mean 
values for SART were computed, where the SART 
rating is “understanding” reduced by the difference 


of “demand” minus “supply” (i.e., SART = 
[understanding - (demand - supply)]). 

ANOVAs on the mean SART ratings found 
significant main effects across the four 
communication modes for all four off-nominal 
events. 

• For the “erroneous data-link clearance” event, 
there was a significant effect found, F(3,24) = 
5.529. A Student-Newman-Keuls (SNK) post- 
hoc test showed that pilots rated the data-link + 
voice and data-link + read-back conditions to be 
significantly higher for situation awareness than 
the data-link only and data-link only + intent 
conditions. 

• The same significant differences in SART ratings 
by condition was found for the “approach runway 
incursion” event, F(3,24) = 7.682. 

• However, SNK post-hoc tests on the significant 

main effects for both the “taxiway conflict” 
(F(3,24) = 13.496), and “data-link clearance 

received for other aircraft” (F(3,24) = 7.362) 

events, evinced that the data-link + read-back 
condition was rated significantly higher for 
situation awareness compared to the other three 
experimental conditions. 

• In comparison, the subjective SA data for the 
corresponding nominal trials showed a significant 
main effect for SART, F(3,95) = 40.675. A SNK 
post-hoc test revealed three subsets: (a) data-link- 
only (1.54), (b) data-link + intent (3.46) and data- 
link + voice (3.92) and (c) data-link + read-back 
(10.25). Therefore, pilots rated the read-back 
only condition to be significantly higher for 
situation awareness than the three other 
conditions and the data-link only to be the worse 
concept for situation awareness during the 
nominal trials. 

Situation Awareness - Workload Dominance 
(SA-SWORD) Scale. An ANOVA revealed a 
significant main effect for communication mode for 
SA-SWORD, F(3, 69) = 40.601. Post-hoc test 
showed that pilots rated the data-link + read-back 
condition to be significantly higher for situation 
awareness than the other three conditions. 



Mental Workload 

NASA Task-Load-Index (NASA-TLX). The 
post-run work was assessed using the NASA TLX 
scale, which is comprised of 6 validated sub-scales 
that are thought to represent the underlying 
construct (rated 0 - 100). An ANOVA found no 
significant main effect for NASA-TLX for any of 
the four off-nominal trials, p > 0.05. The four 
communication modes were found to be equal in 
perceived mental workload as rated by the NASA- 
TLX scale. However, for the nominal trials, there 
was a significant main effect for NASA-TLX, 
F(3,95) = 11.719. SNK post-hoc tests revealed 
three subsets where the pilots rated the data-link + 
read-back condition (23.58) to be significantly 
lower for workload than the other three display 
conditions. The data-link + intent (35.32) and data- 
link + voice (43.31) were both rated significantly 
lower than data-link (54.07). 

Subjective Workload Dominance 
(SWORD) Scale. An ANOVA revealed a 
significant main effect for communication mode for 
SWORD, F(3,69) = 68.169. Post-hoc comparisons 
evinced that pilots rated the data-link + voice (data- 
link as redundant to voice) significantly higher in 
mental workload on the SWORD post-experiment 
paired-comparison scale compared to the other 
three conditions which did not differ statistically 
from each other (p > 0.05). 

Communication Awareness 

Questionnaire Responses. Significant main 
effects were found for communication mode for 
post-run questionnaire item for communication 
awareness for the off-nominal events of “taxiway 
conflict”, F(3,24) = 7.527, and “approach runway 
incursion, F(3,24) = 8.842. SNK post-hoc tests 
revealed that the data-link + read-back condition 
was rated significantly higher for communication 
awareness than the other display conditions which 
did not differ for this measure. 

Analyses for qualitative reports of 
communication awareness taken during the nominal 
trials also revealed a significant communication 
mode main effect, F(3,95) = 44.49. SNK post-hoc 
test showed three subsets: (a) data-link only was 
rated lowest for awareness of aircraft clearances 
and party-line information, (b) data-link + intent 
and data-link + voice was rated higher than data- 


link only but poorer for communication awareness 
compared to data-link + read-back, and (c) data-link 
+ read-back was qualitatively rated the highest for 
the construct than the other three communication 
modes. 

Post-Experimental Paired Comparison. An 

ANOVA revealed a significant main effect for 
communication mode for communication efficacy, 
F(3,69) = 29.635. Post-hoc Least Significant 
Difference (LSD) comparisons showed that pilots 
rated the data-link + voice and data-link only 
conditions to be significantly lower for 
communication effectiveness than both the data- 
link + Intent and data-link + read-back only 
conditions. There were no significant differences 
between the data-link + Intent and data-link + read- 
back only conditions. 

Off-Nominal Event Detection 

Erroneous Data-link Clearance. There 
were 3 off-nominal events per experiment condition 
across the 12 flight crews . Ten of the flight crews 
correctly recognized the erroneous data-link 
clearance (the inclusion of “Mike Four” in the taxi 
clearance) and contacted ATC to query the 
clearance and/or reject the clearance. The two 
instances where the erroneous clearance was not 
detected occurred for the data-link only condition. 

Data-Link Clearance Received for Other 
Aircraft. All the flight crews with either the data- 
link only or data-link + intent condition failed to 
detect the data-link clearance received intended for 
another aircraft and accepted the new clearance 
despite being within 1 nm on final. Upon taxi, 
these flight crews realized that the taxi instructions 
resulted in a route disconnect from the active 
runway to the ramp area and contacted ATC to 
request new taxi instructions. However, none of the 
six flight crews with data-link + voice or data-link + 
read-back accepted the data-link clearance and 
correctly recognized that the data-link message was 
not specific to their aircraft. These flight crews 
ignored the CDPLC message and contacted ATC to 
query the message after turn-off. 

Taxiway Conflict. The data showed that 
none of the flight crews - irrespective of the 
experimental condition - experienced a taxiway 
conflict. Because all the conditions had a HUD 
with a forward-looking-infrared (FLIR) enhanced 



vision system, the Captain was able to detect the 
traffic taxiing toward them on the same taxiway. 
The results showed the significant advantage that an 
enhanced vision system provides for low-visibility 
operations, especially in this scenario. 

Approach Runway Incursion. For the data- 
link only condition, there was one runway incursion 
(33% of the flight crews) that occurred because the 
flight crew was entirely unaware that an aircraft 
was in position-and-hold awaiting take-off 

clearance when the aircraft landed on the active 
runway. For the data-link + intent condition, the 
flight crews were able to adequately recognize and 
avoid the incursion but were slow in identifying the 
evolving situation and reported significant 

workload in interfacing with the traffic intent data 
and determining what the aircraft was intending to 
do. In contrast, the flight crews in both the voice 
conditions began discussing the aircraft on the 
active runway upon hearing the clearance and then 
recognizing that the incursion aircraft had not 
taken-off and had ample time to discuss options, 
initiate a go-around, and contact ATC as to their 
situation. 

Experiment Two Results 

Detailed data analyses for Experiment Two 
are still ongoing and are not available in time for 
publication herein. The off-nominal event detection 
results are discussed in the following. 

Off-Nominal Event Detection 

ADS-B surveillance failure. There was a 
significant difference across the three display 
conditions in terms of detection and recognition of 
an ADS-B surveillance failure. 

All flight crews detected the ADS-B failure in 
the own-ship route and surface traffic with intent 
display condition. In all cases, the first officer 
noticed the absence during selection of aircraft on 
the surface map. Pilot de-brief reports suggest that 
these pilots were actively using the surface map 
display concept because of the presence of valuable 
and useful graphical traffic intent information. The 
loss of surveillance data in this case was obvious. 

One of four flight-crews failed to detect the 
missing aircraft, despite the presence of traffic icons 
in the own-ship route and surface traffic display. 
Often the Captain noticed the aircraft in the HUD 


FLIR and would remark on the absence of the 
aircraft on the surface map. In one case, neither 
pilot noticed the anomaly. 

Three of the four flight-crews with the own- 
ship only surface map displays did not detect the 
ADS-B failure. The reason was due mostly to the 
lack of traffic icons on the surface map. 

In the later two cases, the use of textual 
display of traffic intent data required significant 
cognitive overhead to select and interpret 
information and required a manual selection process 
to get traffic state and intent data. The crews 
consequently did not often use this capability to its 
fullest extent. This reduces the amount of time 
spent evaluating the surface map display and noting 
changes and discrepancies in the data presented. 

Taxi aircraft blunder. In this off-nominal 
scenario involving a runway incursion during own- 
ship departure, only one of the twelve flight-crews 
failed to notice the blunder. The data suggests that 
this is largely owing to the use of the FLIR to detect 
the aircraft. The flight-crews reported significant 
value from the HUD FLIR in preventing this 
conflict and also, the supplemental utility of hearing 
voice communications in which they were made 
aware that an aircraft was holding short down the 
runway. Additionally, the depiction of traffic on 
the surface map was observed to significantly 
enhance situation awareness although it was 
primarily used to confirm the detection of the traffic 
in the HUD FLIR. 

Non-participating vehicle on surface. This 
off-nominal scenario tested whether flight-crews 
would fail to notice a vehicle which was not visible 
on the surface map. The scenario was made more 
challenging because the fire truck had stopped just 
beyond a taxiway intersection. The HUD field-of- 
view would not provide much detection. 

All flight crews were able to detect the fire 
truck suggesting that the presence of a surface map 
display head-down or the near-domain information 
presented on the HUD did not distract the crew 
from the task of scanning OTW for obstacles and 
traffic. All the flight-crews detected the vehicle 
OTW with the exception of two flight-crews who 
noticed the fire truck in the HUD FLIR during taxi 
out of the ramp area. 



Controller mistake clearing own-ship 
across active runway. In this scenario, own-ship 
was cleared to cross the active runway but, after 
several ATC communications, ATC also cleared 
another aircraft to take-off. All flight-crews were 
able to detect the potential incursion situation and 
stop the aircraft. 

Two of the four flight-crews, using the own- 
ship only display, were not able to stop short of the 
hold-line. This is technically a runway incursion. 
All flight-crews with the other two display 
conditions were able to stop short of the hold-line. 

The majority of these flight-crews detected 
the potential conflict from the party-line radio 
broadcast of the erroneous clearance for the other 
aircraft to take-off. They were also able to verify 
this situation by evaluating the surface map display 
if it included traffic. The lack of surface traffic 
reduced the lead-time available to the flight crew to 
mitigate the potential runway incursion. 

In this scenario, the HUD and its enhanced 
vision capability was not useful because the 
incursion aircraft was significantly outside the field- 
of-view of the HUD. 

Conclusions 

The research objective was to evaluate the 
flight deck impacts of different data-link 
implementations that may be considered as options 
for ATC-pilot communications for NextGen. 

Across all off-nominal conditions, the results 
demonstrated that data-link only is not an optimal 
solution for these operations and corroborates 
previous research which advises caution for a 
voice-by-exception system because of the 
significant concerns for reduction of party-line and 
situation awareness on the part of the flight crew. 
The voice-by-exception experimental condition was 
almost universally the highest workload and lowest 
situation awareness condition. The off-nominal 
results confirmed the finding that voice for routine 
communications significantly enhances situation 
awareness for terminal maneuvering area operations 

The subjective and off-nominal data showed 
that the data-link + read-back condition was clearly 
the best option across almost all the dependent 
variables tested. By requiring pilot read-back of 
CPDLC, flight crews responded that they were able 


to retain the party-line broadcast awareness because 
they knew what the other traffic was “intending” to 
do while at the same time reducing the voice 
congestion based on the specified traffic density 
and scripted clearances and pilot responses used in 
present study. This also significantly freed up the 
frequency to allow the flight crews to more easily 
contact the controller if necessary (e.g., non-routine 
communications). 

The off-nominal detection results from 
Experiment Two further supplements the finding of 
Experiment One. The results posit that a surface 
map display with traffic significantly enhances 
situation awareness and would be a significant 
complement to a data-link + read-back only 
approach to data communication and CPDLC. For 
4DT operations, the use of graphical depictions of 
4DT RTA guidance and conformance compliance 
seems to further enhance flight-crew capability 
although data analyses to date make any 
conclusions premature. 

Overall, the results demonstrate the 
substantial importance of providing some party-line 
capability for any new ATC-pilot data 
communication system. The conclusion drawn here 
is that a promising solution may be the use of 
mandatory pilot read-back of CPDLC data-link 
routine clearances in the terminal maneuvering 
area, particularly with a surface map with traffic 
display, and the use of voice communication for all 
non-routine and exigent communications; however, 
the effect of such an approach on the controller has 
yet to be evaluated and the experiment did not 
specifically evaluate controller-pilot interactions. 
The conclusion matches that observed during the 
CASCADE D-Taxi trials at Brussels [22] which 
recommended the use of mandatory pilot read-back 
as, “...effective mitigation means against erroneous 
or non-intended clearance details in the sent 
message” (p. 83) and that a flight-deck display of 
ground traffic would significantly enhance situation 
awareness for data communications in the airport 
movement area. 

Future Directions 

Although the issues of “data-link only” 
have been well-documented in the literature for 
aircraft operations typical of today’s system, this 
“voice-by-exception” data-link protocol continues 



to remain the primary communications end-state. 
The conclusions drawn from the results presented 
herein mirror other research and share the concerns 
for implementation of a safe and effective “data- 
link only” system for the terminal maneuvering 
area. Although voice-by-exception permits voice 
communication - and requires voice for non-routine 
communications - research suggests that there are 
ample situations and opportunities in which a data- 
link only approach may reduce safety margins. The 
use of mandatory pilot read-back of CPDLC data- 
link routine clearances in the terminal maneuvering 
area appears promising and will be explored further. 

This research has also demonstrated the 
utility of flight-deck displays of traffic information 
for surface operations for enhancing situation 
awareness and safety and mitigation of human 
error. This work will continue to evaluate 
methodologies and define whether the visual 
display of traffic intent and other information might 



possibly provide mitigation for the safety concerns 
within a voice-by-exception environment. 


Figure 9 - Runway Incursion Alerting During 
Approach 

Finally, the present results demonstrate that 
flight crews were able to adhere to required-time- 
of-arrival clearances without significant effort even 
under reduced visibility conditions and NextGen- 
like traffic densities. Results from the traffic intent 
research showed that graphical presentation of own- 
ship and other traffic intent significantly enhanced 
situation awareness and conflict detection; however, 
full data analyses are not completed at time of 


publication. Future research will focus on 
optimization of 4DT algorithms, advanced display 
technologies, and conflict detection and alerting 
(Figures 9-10) to meet these and other challenges to 
NextGen operations. 



SURFACE TRAFFIC 

DIST - AIRCRAFT 

1) O.OS AA4IQ3- RJI45 

2) 008 UA592 - B7S7 

3) 0.17 UA3I7 - B737 

4) 0 20 NWQ200- CRJ200 

5) 0.27 UA490 - B747 

6) 0.32 UA784 - B777 

7) 0.37 UA0196- RJ145 

8) 0 38 UA573 - B757 

9) 0 42 UA0153- RJI45 


NONE LI ■ 


Figure 10 - Taxi Conflict Alerting 
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